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DETAILED ACTION 



1 . This is the First Action on the Merits for US Patent Application 10/713,130, which 
is a divisional of US Patent Application 09/378,585, now US patent 6,690,881, filed 20 
August 1999, and claims foreign priority to JP 10-237312 and JP 10-237311, filed 24 
August 1998. Currently, claims 11-24 and 35-68 are pending. In the preliminary 
amendment filed 24 November 2003, claims 1-10 and 25-34 were cancelled. 



Priority 

2. Acknowledgment is made of applicant's claim for foreign priority under 35 
U.S.C. 1 19(a)-(d). The certified copy has been filed in parent Application No. 
09/378,585, filed on 20 August 1999. 

Specification 

3. The disclosure is objected to because of the following informalities: page 3 of the 
specification incorrectly states that MPEG-1 video has a minimum "frame rate" of 29.97 
MHz. 

Appropriate correction is required. 



Claim Objections 

4. Claim 11 is objected to because of the following informalities: in the first line of 
the fifth indentation of the claim, applicant has inadvertently not replaced the word 
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"video" with "picture", consistent with the remainder of the claim. Appropriate correction 
is required. 

Double Patenting 

5. Applicant is advised that should claim 52 be found allowable, claim 66 will be 
objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two 
claims in an application are duplicates or else are so close in content that they both 
cover the same thing, despite a slight difference in wording, it is proper after allowing 
one claim to object to the other as being a substantial duplicate of the allowed claim. 
See MPEP § 706.03(k). 

Claim Rejections - 35 USC §112 

6. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

7. Claims 14, 15, 38, 39, 55, and 56 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. 

8. Claim 14 recites the limitation "the multiplexed data" in the fourth line of the 
claim, and claim 15 recites the limitation "the multiplexed data" in the third and fifth lines 
of the claim. There is insufficient antecedent basis for this limitation in the claims. 
Claim 13 recites a process of multiplexing an encoded picture signal and an encoded 
audio signal, but claims 14 and 15 are dependent directly on claim 11, not claim 13. 
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Similarly, claims 38 and 39 also recite "the multiplexed data", but are dependent on 
claim 35, not claim 37, and claims 55 and 56 are dependent on claim 52, not claim 55. 

Claim Rejections - 35 USC § 103 

9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

10. Claims 11, 12, 14, 16, 17, 20, 23, 24, 35, 36, 38, 40, 41, 44, 47-53, 55, 57, 58, 
61, and 64-68 are rejected under 35 U.S.C. 103(a) as being unpatentable over US 
Patent 6,1 1 1 ,604 A (Hashimoto et al.) in view of US Patent 6,148,031 A (Kato). 

Hashimoto et al. teaches a digital camera. Regarding claims 11, 35, 52, and 66, 
figure 8 of Hashimoto et al. shows a block diagram of the camera. Image 
photographing section 6 comprises lens 7, lens opening 8, imaging element 9, and filter 
10. The analog input image signal is converted in analog/digital converter 4 and further 
processed in DSP 11 (column 6: lines 40-61). This corresponds with the claimed 
"photographing means". Audio signals are input into microphone 1 and output through 
amplifier/filter 2a to analog/digital converter 4 (column 6: lines 18-26). This corresponds 
with the claimed "audio inputting means". Image data compression/expansion circuit 12 
encodes the images from DSP 11 in a format such as JPEG or MPEG (column 6: line 
62-column 7: line 2). This corresponds with the claimed "video encoding means" that 
performs the steps of "encoding the video signal" in the two encoding methods in claims 
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35, 52, and 66. Figure 1 1 of Hashimoto et al. illustrates the process for capturing video 
and information. When the user presses the shutter button, a first picture with 
associated audio is captured. Image and audio files are stored in memory card 16, and 
an association file is written to link the image and audio files together (column 9: lines 
46-54). The association file may be a container file for a still image, a still image with 
audio data, or a moving image with audio data (column 10: lines 1-8). However, in 
Hashimoto et al., no clear distinction is made between various encoding modes for the 
recorded pictures. In the example given in column 9: line 55-column 10: line 4, audio 
may be associated with a still JPEG image, or a moving MJPEG image, or an MPEG 
image (column 6: line 65), which was known in the art at the time the invention was 
made to incorporate sound data. 

Kato teaches an image processing system in a digital camera. Regarding claims 
11 and 35, in Kato, in a continuous imaging mode, input images are initially recorded in 
real time in an intra mode as a succession of JPEG images, and stored in memory 20 
(column 3: lines 41-47). In a still image mode, the input image is recorded in memory 
20 as a single JPEG image (column 3: lines 47-53). After recording is finished, system 
control circuit 26 re-encodes the recorded series of intra images in an inter-frame 
compression mode (column 3: lines 54-63). This system control circuit corresponds 
with the claimed "controlling means" of claim 1 1 , and the selection of a still image mode 
or a motion image mode in Kato corresponds with the claimed recording mode selection 
in claims 35, 52, and 66. 
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Hashimoto et al. discloses the claimed invention except for encoding pictures 
according to two different encoding methods. Kato teaches that it was known to encode 
motion image data in a separate format than still image data. Therefore, it would have 
been obvious to one having ordinary skill in the art at the time the invention was made 
to modify the camera of Hashimoto et al. to re-encode pictures having a temporal 
aspect, such as pictures with associated sound, as inter-frame encoded images after 
encoding, as taught by Kato, since Kato states in column 2: lines 16-34 that such a 
modification would enable the final recording to be achieved with higher compression 
than with intra pictures alone, while maintaining the ability for a user to record a high- 
quality still image during the motion image recording process. 

Regarding claims 12, 36, and 53, in Kato, "the JPEG standard is used in the still 
image compression and the MPEG standard is used in the moving image compression" 
(column 9: lines 12-14). 

Regarding claims 14, 38, and 55, figure 12 of Hashimoto et al. shows video files 
and audio files stored in separate areas of memory card 16. 

Regarding claims 16, 40, and 57, figure 5 of Kato shows a compressed video 
encoder including DCT circuit 107, quantizing circuit 108, and variable length coding 
circuit 115 (column 5: line 60-column 6: line 24). 

Regarding claims 17, 41, and 58, figure 12 of Hashimoto et al. illustrates audio 
and video files stored in the memory as having headers. 
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Regarding claims 20, 44, and 61, Kato temporarily stores incoming data on first 
memory 20, and after re-encoding, permanently stores the data on second memory 22 
(column 4: lines 45-55, "the second memory 22 is the final storage medium"). 

Regarding claims 23, 47, and 64, in Hashimoto et al., image data compression 
circuit 12 may also perform image decoding (column 6: lines 62-66), and so 
corresponds with the claimed "video decoding means". The decoded video signals may 
be further processed in DSP 1 1 (column 6: lines 58-61 ), which outputs a video signal 26 
to a display such as an LCD viewfinder (not shown in figure 8). This display 
corresponds with the claimed "displaying means". Digital audio signals may also be 
decoded in audio data compression/expansion circuit, transmitted to D/A converter, 
amplified and filtered in amplifier 2b, and output in output stream 26 to speaker 32 
(column 5: lines 17-39). This corresponds with the claimed "audio outputting means". 
This process of reading data stored in memory card 16 (column 7: lines 34-50), like all 
other processes of the camera of Hashimoto et al., is controlled by CPU 23 (column 7: 
lines 15-16), which corresponds with the claimed "controlling means". 

Regarding claims 24, 48, and 65, the CCD in Hashimoto et al. has a resolution of 
768 x 480 pixels (column 6: line 44), and Kato inputs images at a resolution of 720 x 
480 pixels, in accordance with the NTSC standard (column 4: line 15), and produces an 
output of 320 x 240 pixels (column 4: line 24), in accordance with the CIF format. 
Although neither Hashimoto et al. nor Kato et al. record pictures at the VGA 640 x 480 
pixel standard, it would have been an obvious matter of design choice to modify the 
image sensing portion of camera of Hashimoto et al. or of Kato to produce 640 x 480 
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pictures, since it has been held that a change in size of a component is generally 
recognized as being within the level of ordinary skill in the art. See In re Rose, 105 
USPQ 237 (CCPA 1955). 

Regarding claim 49, in Kato, as shown in figure 1, system control circuit 26 
controls both the image compression circuit 18, first memory 20, and second memory 
22 (column 3: lines 41-63). This system control circuit corresponds with the claimed 
"controlling means". 

Regarding claims 50 and 67, in Hashimoto et al., incoming image data from a 
camera is processed in noise reduction circuit 10 and DSP 11 (column 6: lines 40-61), 
and incoming audio data from a microphone is processed in amplifier/filter 3a (column 
6: lines 18-21). 

Regarding claims 51 and 68, in Hashimoto et al., figure 14 illustrates the 
flowchart for transmitting and receiving data from the camera to an external device 
(column 10: line 41-column 11: line 42). Data from the memory card is transferred to 
FIFO 13 (column 11: lines 25-29), and transmitted to an external device via interface 
circuit 27 (column 7: lines 1-36). Like every other process in the camera of Hashimoto 
et al., this process is controlled by CPU 23 (column 7: lines 15-16), which corresponds 
with the claimed "controlling means". 

11. Claims 13, 15, 18, 19, 37, 39, 42, 43, 54, 56, 59, and 60 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Hashimoto et al. in view of Kato as applied to 
claim 11 above, and further in view of ISO/IEC 11172-1 (MPEG-1 Part 1). Claims 13, 
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37, and 54 of the present invention is directed to multiplexing an encoded picture signal 
and an encoded audio signal. Hashimoto et al. teaches "combining" a video file and an 
audio file (column 11: lines 34-42), but does not give details about how this is 
accomplished. 

MPEG-1 Part 1 defines the system coding layer of an MPEG-1 coded data 
stream, in which audio and video data streams are multiplexed (forward). Regarding 
claims 13 and 37, Section 1-A.6.3 illustrates a sample multiplexing of a stream having 
one video and one audio stream. The stream is divided into packs, each of which has a 
header and three packets, each of 2048 bytes. First, 13 video packets are transmitted 
to ensure successful buffering. Then, an audio packet is placed for every 6.25 video 
packets. Section 1-A.6.9 shows an extended sample multiplexed data stream. Here, a 
second audio packet is placed between the twentieth and twenty-first video packets. 

Hashimoto et al., in combination with Kato, discloses the claimed invention 
except for multiplexing an audio and picture signal. MPEG-1 Part 1 teaches that it was 
known to produce a multimedia datastream by multiplexing packets of audio and video 
data. Therefore, it would have been obvious to one having ordinary skill in the art at the 
time the invention was made to implement the combination of a video and audio file in 
Hashimoto et al. as a multiplexing operation, as taught by MPEG-1 Part 1, since 
MPEG-1 Part 1 states in the Introduction that such a modification would allow for 
synchronized playback of audio and video data without having to buffer an entire sub- 
stream. 
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Regarding claims 18, 42, and 59, in MPEG-1 Part 1, excluding the first pack 
containing a special header and padding, all packs contain exactly 12 + 3 x 2048 = 
6156 bytes (Section 1-A.6.3). 

Regarding claims 15, 39, and 56, in Kato, as mentioned previously, video data is 
first stored in first memory 20 and then transferred to second memory 22 (column 3: 
lines 54-63). This corresponds with writing multiplexed data to memory, reading the 
multiplexed data from memory, and recording multiplexed data on a recording medium. 
Additionally, in Kato, during the recording of a moving image, a still image from the 
sequence of moving images may be additionally transferred from the first memory to the 
second memory as an intra picture in an independent process of the moving picture 
recording (column 4: lines 1-10). This corresponds with encoding a video signal in the 
"first" encoding method, writing the signal to the memory, reading the signal from the 
memory, and recording the signal to the recording medium. 

Regarding claims 19, 43, and 60, an MPEG-1 pack, containing 3 packets, is 
designed to have a pack rate of 29 Hz, or 1 frame per pack (Section 1-A.6.3). 

12. Claims 21, 22, 45, 46, 62, and 63 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hashimoto et al. in view of Kato as applied to claims 11, 35, and 62 
above, and further in view of US Patent 6,327,423 B1 (Ejima et al.). Claims 21, 22, 45, 
46, 62, and 63 are directed to specific operations of causing a camera to perform a 
capture for a certain time. Hashimoto et al. teaches taking pictures when a shutter 
button is pressed (column 7: lines 20-24), and Kato teaches operating a keyboard to 
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issue image taking commands (column 3: lines 41-53). However, the above references 
do not teach operation for a "time period" to generate data. 

Ejima et al. teaches a camcorder that records sound data. Regarding claims 21 , 
45, and 62, figure 14 is a flowchart illustrating one embodiment of the sound recording 
control process of Ejima et al. At step S1, CPU 39 determines if a release switch 10 is 
pressed, and if it is, the image recording process begins at step S2 (column 15: line 64- 
column 16: line 3). At step S3, the sound recording process is started, and at step S4, a 
"REC" display is shown on a viewfinder to indicate that sound is being recorded (column 
16: lines 4-1 1 ). At step S5, after 10 seconds have passed, the sound recording process 
stops (column 16: lines 11-16, 34-40). However, if a sound recording switch is pressed 
within 10 seconds at step S6, sound recording continues (column 16: lines 14-23, 44- 
50). The sound recording then ends when the sound recording switch is released at 
step S20 (column 16: lines 23-50). Then, sound recording switch 12 corresponds with 
the claimed "operating means", and the time period in which the sound recording switch 
is pressed corresponds with the claimed "timing means". 

Hashimoto et al., in combination with Kato, discloses the claimed invention 
except for encoding audio during the pressing of an operation means. Ejima et al. 
teaches that it was known to perform sound recording while a sound recording switch is 
pressed. Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to incorporate the sound recording switch of Ejima et 
al. into the camera of Hashimoto et al. or Kato, since Ejima et al. states in column 1 : line 
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60-column 2: line 20 that such a modification would allow the timing of a sound 
recording to be independent of the timing of its associated video recording. 

Regarding claims 22, 46, and 63, in Kato, if sound recording switch 12 is not 
pressed, then release switch 10 corresponds with the claimed "operating means", and 
the ten seconds is the "predetermined time period" in which audio is encoded. 

Conclusion 

13. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. US Patent 4,188,642 teaches a CCD for an imaging apparatus 
read in a single frame and a multiple frame mode. US Patent 4,527,205 teaches a 
camera that takes still pictures and motion pictures. US Patent 4,546,390 teaches a 
combination still camera and motion picture camera. US Patent 4,663,669 teaches an 
image sensing apparatus that produces higher-quality images in a still mode than in a 
movie mode. US Patent 4,714,963 (Vogel) teaches a still camera that produces a 
moving image for previewing in a viewfinder. US Patent 4,816,928 teaches an 
electronic camera that records audio data with visual images. US Patent 4,837,628 
teaches an electronic still camera with a motion viewfinder that operates when a shutter 
is half depressed. US Patent 5,206,730 A teaches a digital camera having multiple 
photographing modes. US Patent 5,436,657 A teaches an electronic camera that 
records sound. US Patent 5,440,343 A teaches an image recording system that 
records NTSC motion images and megapixel-level still images. US Patent 5,444,482 A 
teaches a shutter mechanism for a digital camera. US Patent 5,444,483 A teaches a 
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digital electronic camera that records still images and motion images. US Patent 
5,497,194 A teaches a camera that photographs images and records sounds. US 
Patent 5,614,946 A teaches a still camera that records audio and visual signals. US 
Patent 5,815,201 A teaches another aspect of the camera the cited Hashimoto et al. 
reference. US Patent 6,690,881 B1 is the parent application for the present invention. 
European Patent Application Publication 859,511 A1 (Abe et al.) teaches a recording 
system for recording still pictures, moving pictures, and sound. European Patent 
Application Publication 982,950 A3 (Tomita et al.) is the corresponding European 
version of the present invention, and includes a search report. Japanese Patent 
Application Publication 02-280484 A (Tanabe et al.) teaches a camera that 
synchronizes recorded audio and video data. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner, whose telephone number is (571)272- 
9662. The examiner can normally be reached on MWF from 9:00-6:30, TR from 9:00- 
4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri, can be reached on (571) 272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
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published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



ID. N. W./ 

Examiner, Art Unit 2621 



